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ABSTRACT 



Renton Vocational Technical Institute received funding for 
a special project entitled "Development of an Instrument For 
Prescribing Compensatory Education For Vocational Technical 
Trainees**, Project 75-5110 (125) 08, to develop three pre- 
entry criterion-referenced tests (Skills Checks). These 
Skills Checks are designed 1) to assess an applicant's verbal 
and numerical competencies which are identified as prerequi- 
sites to successful training in the applicant's vocational 
area; and 2) to assist in implementation of rem.edial educa- 
tion based on deficiencies identified by the Skills Checks. 
This goal of developing three pre-entry criterion-referenced 
Skills Checks has been achieved. 

Skills Checks have been developed for vocational training 
areas of Office Occupations, Automotive Mechanics and Machine 
Shop. All Skills Checks have undergone field testing, item 
analysis, measures of validity (when possible) and evaluation 
by the Third Party Evaluator, Dr. Charles Schultz. The Skills 
Checks are being administered to all applicants to these vo- 
cational training areas as part of the application procedure 
of the Institution. Those applicants demonstrating skill 
deficiencies are being provided with appropriate remediation. 

The first year of the development of Skills Checks has gener- 
ated interest and support from students and staff at Renton 
Vocational Technical Institute and in other vocational train- 
ing programs throughout the state. Based on verbalized in- 
terest, the project applied for a continuation grant for FY 76 
Funding was granted to the continuation proposal. It is an- 
ticipated that during the continuation year, the project will 
develop^ an additional six Skills Checks in other vocational 
training areas at Renton Vocational Technical Institute, and 
an additional four Skills Checks in subjects offered through 
other vocational training programs. It is expected that 



the Research Assistant will work in close cooperation with 
individuals from other institutions in order to familiarize 
them with the operation of the Skills Check program. It 
is anticipated that this cooperation with other training 
programs will encourage modifications in the project which 
will increase its generalized applicability throughout the 
state of Washington. 
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STATEMENT OF PROBLEM 



The recent emergence and rapid growth of vocational training 
institutions has given rise to a number of critical problems 
relating to evaluation and guidance of applicants. Often ap- 
plicants do not have sufficient preparation to begin training 
in a vocational program. Yet, due to an ''open-door" policy, 
they are admitted. One of the consequences of this situation 
is that several students, upon entering vocational programs, 
discover that they are incapable of doing the required work, 
begin to flounder, become frustrated and begin to be tardy <, 
absent, and eventually drop-out. Attendance records echo this 
pattern over and over again. 

Vocational training institutions have not ignored this problem. 
They recognize the need for applicants to possess adequate pre- 
requisite skills and knowledge prior to enrollment. Typically, 
applicants are characterized by a wide range of aptitudes, in- 
terests, and achievements. The need to assess aptitudes and 
achievements in some rational manner has_ led to the development 
of instruments and techniques designed to either screen out 
persons exhibiting deficiencies or to identify them so that com 
pensatory instruction can be provided. 

Insight into individual strengths and weaknesses has been 
sought through an examination of background data, interviews, 
and standardized tests. Efforts in the latter instance have 
been hampered seriously by the failure of existing instruments 
to meet three conditions: (1) contain items relevant to voca- 
tional technical instruction, (2) appeal to young and older 
adults, and (3) contain items that are not prejudicial to appli 
cants in that they are culturally laden and/or heavily reliant 
on verbal skills. 

This lack of standardized instruments appropriate to diagnosing 
an applicant's deficiencies or to predicting trainee success ha 



led to a number of situations. Some training agencies simply 
do not use tests while others knowingly misuse measuring instru- 
ments. A few agencies seek empirical combinations of measure- 
meats of Individual traits and separate factors of intelligence 
into instruments designed and weighted to predict success in 
specific areas of training within their institutions. 

While no one familiar with current psychological research can 
argue against trait and factor theory as a basis of selection, 
it should not be the only criterion. As Raymond Ross points out, 
''Information on personal characteristics as they relate to vari- 
ous career choice options is a necessary but not sufficient con- 
dition for optimizing career development" (4:73). The central 
underlying assumption of this project is that if the real concern 
of vocational educators is to admit all adults and help them suc- 
cessfully complete training, attention must be directed to iden- 
tifying achievement level. Identification must be in such a man- 
ner as to allow the systematic remediation of relevant learning 
deficiencies through prescribed compensatory training activities 
conducted either prior to or concurrent with job training. 

This project's ultimate objective is the development of tests 
(hereafter referred to as Skills Checks) for specific vocational 
training areas.. Skills Checks would be comprised of items rele- 
vant only to a particular training program. Only skills defined 
as necessary prerequisites would be measured. Information sup- 
plied by a Skills Check would permit vocational technical insti- 
tutes and community colleges to assess and compare an applicant's 
achievement level with the level determined as necessary for en- 
trance and successful participation in the applicant's chosen 
vocational area. Persons exhibiting deficiencies in areas for 
which they request vocational training would be encouraged to 
receive relevant compensatory instruction. 
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It is anticipated that successful identification of skill de 
ficiencies and remediation of those skills will result in re 
duced rates of tardiness, absenteeism and unscheduled with- 
drawals. Successful completions of vocational programs are 
expected to increase. 

Utilizing the principles of criterion-referenced testing, 
(see Criterion-referenced Testing Section), this project's 
ultimate goals are to: 

Develop Skills Checks that identify those students 
demonstrating insufficient competency in one or more 
skills. 

Develop Skills Checks that will provide information 
for prescriptive remedial education, causing instruc- 
tion to consist of only those skills necessary for 
• participation in a given occupational area. 
Design Skills Checks in a manner that will allow .appli- 
cation to vocational programs throughout Washington 
State. 



OBJECTIVE 



To increase the percentage of students who successfully complete 
vocational training programs as a result of the development and 
implementation of pre-entry criterion-referenced Skills Checks 
and remedial education programs which are based on such Skills 
Checks at Renton Vocational Technical Institute from the cur- 
rent estimate of no students tested per selected class to all 
potential enrollees per selected class by June 30, 1975. 

Specific Objectives 

1. Select project staff. 

2. Complete a thorough review of the literature by 
the Research Assistant. 

3. Identify basic entry-level verbal-numerical skills 
required for success in three selected vocational 
training programs. 

4. Develop test instruments which will test basic 
skills relevant to the three selected vocational 
training programs. These tests will be adminis- 
tered to all applicants for each vocational train- 
ing program. 

5. Prescribe specific remedial instruction in areas 
of demonstrated weakness. 

6. Provide the prescribed instruction for a minimum of 
ten persons in each of the three training areas. 

7. Assess the effectiveness of the prescribed instruction 
by analyzing the rate of success of those students who 
receive compensatory instruction and subsequently enter 
and remain in vocational training programs. 

8. Prepare final report. 

A brief schedule for completion of the above objectives is 
given on the following page. 
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TIME LINE 



October 



1. Selection of project staff 

2. Review of literature 

3. Identification of course 
areas to be utilized 



November, December 



1. Analysis of course content 

2. Development of rough draft 
of course tests 



January 



1. Field test of rough draft 
of tests 

2. Individuals for case studies 
identified 



February, March 



1. Remediation provided 

2. Case studies begun 

3. Tests evaluated 



April, May 



1. Tests revised, utilized, 
and evaluated 

2. Subsequent performance in 
vocational training areas 
of those students who have 
received remedial education 
observed 



June 



Final report prepared 
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STAFF 



Project Director 



Dr. Robert C. Roberts 

Assistant Superintendent, Renton School District #403 
Administrative Director, Renton Vocational Technical Institute 
5000 Northeast 4th Street 
Renton, WA 98055 



Project Assignment 

The Project Director is responsible for satisfactory completion 
of the project. While not involved in the daily conduct of the 
project he provides general guidance for project direction and 
staff cooperation. 
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Principal Investigator 



Delight C. Willing 

Coordinator: Supplemental Support Services 
Renton Vocational Technical Institute 
3000 Northeast 4th Street 
Renton, WA 98055 

Background: B,A. - Carlton College, Teaching Certificate- 
Ottawa College, M.A. (July '75) - Seattle Univ. 

Project Assignment 

The Principal Investigator will insure that scheduled project 
activities are conducted in accordance with the project plan. 
It is estimated that 440 hours will be required for this task. 
The Principal Investigator will accomplish the following speci- 
fic tasks: 

1. Selection of other project staff. 

2. Supervision of review of the literature. 

3. Supervision of the training areas to be used for the 
research project. 

4. Supervision of the analysis of the course content of 
the selected vocational training areas. 

5. Supervision of the development of the specific matrix 
of competencies for each selected vocational training 
area. 

6. Supervision of test development. 

7. Supervision of field test. 

8. Supervision of identification of individuals for case 
studies . 

9. Prescription of specific remedial instruction in areas 
of weakness for identified individuals. 

10. Supervision of remedial education. 

11. Supervision of case study preparation. 

12. Supervision of test review and revision. 

13. Supervision of the preparation of the final report. 
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Research Assistant 

Craig Mueller 
Research Assistant 

Renton Vocational Technical Institute 
3000 Northeast 4th Street 
Renton, WA 9805S 

Background: B. A. - Univ. of Washington, M.A. - Western Wash. 
State College 



Project Assignment 

The Research Assistant will conduct much of the research and 
maintain the records for the project. It is anticipated that 
the Research Assistant will have background in testing, through 
courses in education, psychology, or sociology. In addition, 
the Research Assistant will be expected to maintain detailed 
records during the duration of the project to be utilized in 
preparation of a complete final report. The Research Assistant 
will accomplish the following specific tasks: 

1. Conduct review of literature. 

2. Become familiar with criterion-referenced testing, 
readability analyses, performance objectives, and 
other methods of course content analysis. 

3. Analysis of the course content of the selected voca- 
tional training areas. 

4. Development of the specific matrix of competencies for 
each selected vocational training area. 

5. Development of specific criterion-referenced tests. 

6. Conduct of field test of tests developed, including 
actual testing of prospective vocational trainees, 
test scoring, and test analysis. 

7. Identification and recommendation of individuals for 
case studies . 

8. Preparation of individual case studies. 

9. Completion of test review and revision. 
10. Preparation of the final report. 
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Third Party Evaluator 



Dr. Charles Schultz 

Supervisor - Examination Standards 

Washington State Dept. of Personnel 

600 S. Franklin St. 

Olympia, WA 98501 

Background: B.S. M.S. Ph.D. - Univ. of Washington 



Project Assignment 

I. Provide continuous evaluation of project progress through 
critical review and redirection in following areas: 

A. Development of measurement objectives 

1. Design of content analysis methods 

a. Review of curriculum material 

b. Interview of instructor, students 

c. Observation of problem areas 

2. Avoidance of irrelevant content 

B. Review of Test Segments 

1. Item construction 

2. Balance of item in a test (length of segments) 

C. Reliability 

1. Methods for building in reliability 

2. Methods for assessing reliability 

D. Validity 

1. Content validity 

a. Evaluation of content domain definition 

b. Adequacy of test as measure of domain 

2. Empirical validity 

a. Value of the test in improving performance 

b. Quasi-experimental design 

E. Test Characteristics 

1. Interpretation of test scores 

2. Usefulness for diagnostic testing 

II. Prepare evaluation of final report, to be attached to 
Project Final Report, ascertaining degree of success 
with which project met its objectives. 
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DISSEMINATION 



The final report, to be prepared as part o£ the continuation 
project, will be transmitted to vocational technical institutes, 
community colleges, and other significant institutions in the 
Washington vocational delivery system. It is expected that 
interest will be such as to warrant the conduct of several 
short-term workshops. If so, funds will be requested through 
the Research Coordinating Unit for this inservice training. 
These services will be provided by June, 1976. 

Plans call for the development of a non- technical report to be 
submitted for publication to several journals that are widely 
read by vocational educators. 
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REVIEW OF THE LITERATURE 



A review of the literature was carried out to familiarize pro- 
ject staff with current research and to locate any research 
which would make this project seem redundant. The review in- 
cluded a search of the ERIC files from 1967 to the present, 
using the following descriptors: 

Vocational Aptitude 

Vocational Counseling 

Vocational Development 

Vocational Education 

Adult Vocational Testing 

The literature review revealed that administrators, counselors, 
and educators have expended much energy attempting to obtain 
reliable vocational guidance information. It was observed that 
regardless of whether interests ran from '*self-concept" (Brian: 
1973), CWorthington:1973) to "student autonomy in decision- 
making" CRichardville:1972) , practitioners overwhelmingly re- 
lied upon standardized aptitude-type instruments. Even when 
sought information required knowledge of an applicant's com- 
petencies, aptitude or combinations of aptitude-achievement type 
instruments were used. The overwhelming majority of sources 
dealt with the aptitude and interest tests; little mention was 
made of assessments of cognitive knowledge as related to success 
in vocational training. Sources that were orientated toward cog- 
nitive knowledge (achievement) usually dealt with post-training 
measures of competency and relationship to occupational success. 

Since so many of the materials reviewed focus upon aptitude and 
interest testing, perhaps a note on the difference between apti- 
tude and achievement tests is appropriate here. The difference 
between achievement and aptitude testing is a difference in the 
degree of uniformity of relevant antecedent experience. 
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Achievement tests measure the effects of relatively standardized 
sets of experiences. While aptitude tests measure the effects of 
learning under relatively uncontrolled and unknown conditions, 
achievement tests measure the effects of learning that occurred 
under partially known and controlled conditions. 

Aptitude tests serve to predict subsequent performance. Achieve- 
ment tests represent an evaluation of the individual's status 
upon the completion of training, or current level of knowledge. 
Of course, an achievement test can be used as a predictor when 
different individuals have had the same or closely similar courses 
of study. 

When achievement tests are employed in placement and classifica- 
tion, attention must be given to trait difference. For this 
reason, the use of achievement tests in different areas has been 
advocated. Achievement tests are useful in the identification 
of pupils with special educational disabilities and in the measure- 
ment of progress in the course of remedial work. (Note: This 
material is based on the work of Anne Anastasi.) 

One source that had objectives similar to this project's was... 
"Minnesota Vocational Interest Inventory Training Success Norms" 
by David Pucel and Howard Nelson. This article is one of several 
to come out of Project MINI -SCORE. The authors point out the need 
to identify basic competencies and selected characteristics which 
would be useful to counselors and others in the counseling of 
vocational-technical orientated students as they seek to pursue 
specific vocational-technical curricula. The authors rely upon 
the Minnesota Vocational Interest Inventory (MVII)... an instrument 
composed of 158 triads of brief statements describing the tasks 
or activities involved in a number of different occupations. It 
is at the point of instrument selection that similarities between 
this article and project stop. The MVII is designed in a manner 
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that combines basic competencies and selected characteristics 
into interest patterns. So what the authors are actually deal- 
ing with and measuring are the underlying interests of appli- 
cants. The interest patterns of successful vocational graduates 
were measured to form a norm by which an applicant's interest 
pattern is compared. If an applicant's interest pattern fell 
noticeably out of the normative pattern the applicant could 
be advised that this particular program may not be suitable. 
This type of assessment relies upon the standardized testing 
approach to securing information, ignoring the fact that the 
applicant may or may not have appropriate skills. This source 
is typical of many of the interest-orientated projects designed 
to provide information for decision-making. Directing himself 
to literature dealing with vocational interest inventories, 
Osipow in Karl Kuze's "Overview of Needs, Programs and Imple- 
mentations of Vocational Counseling and Guidance" states, 
"There is a tendency in our culture to make choices consistent 
with interest and hope or even assume that abilities will prove 
to be adequate when, in fact, most of the data indicate or lead 
to inference that abilities far override interests in influ- 
encing vocational attainment" (Kuze 2:70). 

There is also the methodological problem of ignoring, over time, 
changes in interest. It seems unreasonable to compare an appli- 
cant's interest with a tradesman's interest. A tradesman has 
undergone extensive training which could easily effect and change 
interests. Were the interests the same at graduation as at ap- 
plication? What were the interest patterns of successful voc- 
ational graduates when they applied vs. when they graduated and 
began work? If there is no significant difference, one might 
suspect interest patterns at application would be more reliable 
and commensurate with counseling an applicant since these in- 
terest patterns would illuminate the initial interests that 
correlate highly with success in program and occupation. 
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Dr. Margaret Crawford also expresses an interest in assessing 
an applicant's competencies for placement into suitable occupa- 
tional training areas. In her article, "Selection and Guid- 
ance of Students for Technical and Vocational Education" Dr. 
Crawford describes the development of an aptitude test battery 
that is currently in use at Los Angeles Trade-Technical College. 
The battery is a combination of personality and ability criter- 
ion usually measured separately by different standardized tests. 
The reliance of an aptitude- type test battery to obtain infor-. 
mation about abilities is common throughout the literature. Dr. 
Crawford has found fault with this test battery. She states, 
"A further problem is the subjective nature of the personality 
test combined and high weightings of these variables in a bat- 
tery caused by the low intercorrelation between personality 
test scores and ability scores. We need to use caution in 
these areas. Vocational nursing is a good example of this 
problem. Many studies of nursing aptitudes show strong correl- 
ations with the theoretical part of the occupation but little 
correlation with work in the wards. Nursing examinations are 
highly verbal and tend to stress reading comprehension and 
academic abilities" (Crawford 16:66). 

By this statement, Dr. Crawford points out the following: 1) 
The problem of standardized items being culturally laden or 
heavily reliant on verbal skills, 2) The inadequacy of com- 
bining tests into an aptitude test battery to assess abilities, 
3) The need for instruments to better assess abilities in 
relationship to job-related work. This project speaks directly 
to these issues. 

The logic of this project was met in "Basic Skills for Health 
Occupations, Health Service Aide" by Dade County Public Schools, 
Miami, Florida. Dade County Public Schools has designed a 45 
hour, no-credit course to provide selected beginning assistants 
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in Health Occupations with initial skills and background infor- 
mation. Each student is provided with classroom experience in 
basic fundamental and manipulative skills applicable to all 
health services. Although the logic is similar, there are other 
factors which make this article non-amenable to this project's 
needs. There is no mention as to how students are selected for 
this special pre-program course. Whether admission is volun- 
tary, involuntary, or based on skill deficiencies is unknown. 
Dade County is fortunate in being able to set up one program 
providing basic skills applicable to several related occupa- 
tions. The typical vocational training center is represented 
by a diversity of occupations, each requiring different pre- 
entry competencies. 

This article is the only one reviewed that dealt specifically 
with the need and means of providing training in basic compet- 
encies before a student began actual training. Unfortunately 
the success of the course could not be assessed because, at 
this point in time, there was no follow-up comparing students 
who went through the pre-program course with students who did 
not. 

The need for performance tests for vocational placement is 
underlined in "Predictive Testing for Entrance in Vocational- 
Technical Schools" by John Miller. This study focuses on valid- 
" ity of prediction of success-type tests in vocational areas of 
machine shop and electrical wiring. The type of tests developed 
by Miller were problem situations set up to resemble everyday 
situations that tradesmen must be able to handle. The predic- 
tive value of these tests was very high, but this could have 
been due to the population tested. All applicants tested had 
had previous training in these areas in high school. 

Although these tests seem to depart from this project by assess- 
ing more than basic pre-entry skills, i.e., skills taught in 
' class, the author is asking a very important question that is 
usually neglected. Does the applicant have skills to do the 
required coursework? 
O 15 ,- 21 



Another source that advocated performance tests for vocational 
placement was "Rationale For and Development of a Battery of 
Performance Tests for Vocational Students", by William A. Mc- 
Laurin, and others. However, the instrument designed to measure 
performance appears to be largely numerical in content. In its 
conclusion, this article points out that it is further re- 
cognized that in many training programs much of the training 
material is highly verbal in nature. However, if an applicant 
can demonstrate the basic aptitudes required for success in an 
occupation, modification of training programs would be justi- 
fiable and advisable" (McLaurin 16,17:70) 

This suggestion of modifying training programs brings about 
questions of: Given an applicant has the appropriate aptitude 
but is verbally deficient, how is this deficiency to be identi- 
fied and remediated in a manner appropriate to the training 
area? How are the skills required for the success to be iden- 
tified and remediated without requiring the student to study 
extraneous materials? These are questions that the project will 
attempt to resolve. 

Thus, the Review of the Literature demonstrated that in no ways 
were this project's objectives duplicated. Throughout the 
literature there was expressed concern and need for vocational 
guidance and placement information. It was observed that 
interest usually was directed by questions about an applicant's 
aptitudes or competencies other than pre-entry skills. Seldom 
was the question asked, does the applicant have the necessary 
skills to begin coursework? When it seemed prerequisite skills 
were to be dealt with, invariably, there would be reliance 
upon standardized, aptitude- type instruments. It is imperative 
that practioners realize that instruments are designed to 
answer specific questions and meet specific needs. Using 
an instrument to answer questions and meet needs for which 
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it wns not dosiv»iu\l constitutes u misuse of that instrument and 
results in inndequnte information. 

In review this projeet directs itself to the follow i.ng questions: 
What are the pre-entry skills necessary for successful 
participation in a given vocational program? 
How are necessary pre-entry skills to be identified? 
How are deficiencies and necessary remediation to be 
ident i fied? 

This project directs itself to these basic questions. ff success- 
ful, the resulting developed instrument of this project will pro- 
vide much needed information for counseling guidance, and place- 
ment of vocational applicants. 
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CRITERION -REFERENCfiD TESTING 



Katz brings to front an important clement in vocational training. 
"Vocational development may be a continuous process, but... the 
process is enacted through a sequence of choices'* (8:1966). 
All vocational careers require decision-making on the part o£ 
the student, counselors, and vocational educators. Adequate 
decision-making requires reliable and valid information. Select- 
ing a method for obtaining information is analagous to a mechanic 
selecting the tool best suited to do the job. With regard to 
this project, there are two tools one can choose from.... 
standardized tests or criterion -referenced tests. 

As mentioned, there has been a great reliance placed upon stan- 
dardized tests to provide information for counseling and guidance. 
Recently, examiners, counselors and educators have leveled addi- 
tional criticism at the standardized approach to testing. The 
common complaint is that there is a need for certain types of 
information that standardized tests are incapable of providing. 
John Fremer states, "People want something more than they have 
been getting. A common theme in the requests for something 
more is that test users want more information that will be use- 
ful in making decisions" (Fremer 1:73). 

Prediger, in an article, "Converting Test Data to Counseling 
Information" speaks of bridges between data and information. 
Because standardized tests usually rely upon nati<5nal averages 
for individual score interpretation, he refers to standardized 
test data as a norm bridge. He states, "The norm bridge is an 
incomplete bridge since test norms simply permit one to estimate 
standing in some group and do not, per se, indicate the implica- 
tions of this standing" (Prediger 2:71). Klein also lends sup- 
port to the need for a new approach to testing. "Comparatively 
little help has been given to the classroom teacher to diagnose 



individual student needs" (Klein 1:73). Issues of account- 
ability or what does a student know in terms of instructional 
objectives are often ignored. In reaction to this growing 
criticism a new form of testing has evolved. . .Criterion-referenced 
testing. Unlike Standardized tests, this form of testing usually 
does not use national averages or comparison of position in a 
range as correlates of score interpretation. In a recent paper, 
"Standardized vs. Criterion-referenced Testing", Burdette Han- 
sen aptly points out similarities and differences of the two 
testing forms. Listed below are issues one should be familiar 
with before designing a test to meet particular information 
needs. 

The two forms are similar in that both adhere to a base of 
educational objectives. However, standardized test objectives 
are based on what the majority of schools or institutions 
throughout the nation are attempting to achieve. Using a stand- 
ardized form entails the assumption that objectives based on 
national norms met a particular institution's needs... i.e., 
objectives are similar. If this assumption is unreasonable, 
criterion-referenced is probably the better form. "Criterion- 
referenced tests are usually built around the school's own ob- 
jectives" (Hansen 19:74). 

Interpretation of scores received by examinees differ accord- 
ing to what form is used. With standardized tests, a raw score 
is converted into a standard score based on a national norm. 
Standard scores are placed on a bell-shaped curve for evaluation 
of achievement. This practice results in the placement of fifty 
percent of the tested population above the average student and 
fifty percent below. Outcome is more dependent upon method of 
scoring rather than actual scores. Results of criterion- 
referenced tests tend to ignore national standards and bell- 
shaped reporting schemes. "A criterion-referenced measure is 
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essentially one that is deliberately constructed so as to 
yield measurements that are directly interpretable in terms 
of specified performance standards" (Klein 1 : 73) .Hansen 
points out that reporting in terms of the specific objective 
being measured makes it a simple task for a classroom teacher 
to determine whether or not the student knows the objective 
being measured (Hansen: 1974) . 

Thus, there are two important differences between standardized 
and criterion-referenced forms. (1) Standardized tests are 
comprised of items designed to measure objectives generalized 
across populations and institutions * Items for criterion- 
referenced tests are designed to measure objectives specific 
to particular populations and institutions. (2) Standardized 
scoring reports where a student places in relation to all 
other persons who took the test. Criterion-referenced scoring 
is not concerned with the relative position of a student with 
his peer group, but directs itself to the question... does the 
student know the material? 

Since the project's objective is to develop tests that will 
provide information about an applicant's ability to perform pre 
requisite skills specific to a particular vocational program, 
a criterion-referenced approach was adopted. 
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1. There are some instances of standardized tests failing to 
generalize measurability across populations. In particular, 
David Pucel in "The Ability of Standardized Test Instruments 
to Predict Training Success and Employment Success" used six 
instruments. . .General Aptitude Test Battery, Minnesota Voca- 
tional Interest Survey, Sixteen Personality Factor Question- 
naires, Minnesota Importance Questionnaire, Vocational Devel- 
opment Inventory, and Minnesota Scholastic Aptitude Tests to 
measure criteria across six populations. He found, "None of 
the instruments were consistently most highly correlated with 
the same criteria across all three total populations on the six 
curriculum populations. The relationship between an instrument 
and the criteria changed from population to population, imply- 
ing that an instrument that might be most effective in predict- 
ing a criterion in one population may not be the most effective 
in predicting that same criterion in another population." 
(Pucel 2:72). 
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METHODOLOGY , OVERVI EW 



1. The first task required the Principal Investigator to 
assemble a staff composed of a Research Assistant and 
a third-party evaluator. (See Staff section for pro- 
ject assignments). 

2. Upon selection of the Research Assistant, a thorough 
review of the literature was carried out. The initial 
proposal was based upon the assumption that the review 
would not locate any research which would make the pro- 
ject seem redundant. This assumption was substantiated. 

3. Three of the five traditional training service areas that 
are common to a majority of the vocational technical in- 
stitutes and community colleges in the state of Washington 
were identified as subjects for this project. 

4. An analysis of course content for the selected vocational 
training areas was carried out. The analysis included 
readability studies on all materials used in the course 
in order to ascertain the average reading level assumed. 
Numerical skills presupposed in the course content were 
analyzed by an item analysis. In addition to analysis of 
materials, instructors and students were interviewed to 
determine levels of knowledge presupposed in instructional 
content, and student perception of what they considered 

to be necessary pre-entry skills. 

5. A general matrix of competencies for each selected voca- 
tional training area was established by the project staff 
and reviewed by the instructional staff. 

6. Specific Skills Checks, utilizing the course matrices 
developed, were developed for each selected vocational 
training area. Skills Checks were composed of specific 
criterion-referenced items designed to evaluate an appli- 
cant's verbal-numerical skills in relation to the course 
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matrix. Items utilized subject matter drawn from a speci- 
fic vocational training area. 

7. The Skills Checks developed were field tested on all ap- 
plicants to each selected vocational training program. 

8. When specific deficiencies were demonstrated through per- 
formance on the test, the Project Director prescribed 
specific remedial instruction in areas of weakness. If 
deficiencies proved to be great, the applicant was requested 
to master educational objectives before entry into the vo- 
cational training program. If deficiencies were minor, 
such remedial education took place concurrent with voca- 
tional training. If weaknesses were great and the student 
was not receptive to remedial education, the Project Direc- 
tor and vocational counselor had the option of suggesting 
alternative vocational training programs which appeared 
more congruent with the applicant's verbal-numerical skills. 

9. Originally, the project staff was to identify at least ten 
students who demonstrated specific weaknesses among those 
tested in each of the selected vocational areas. Detailed 
case studies involving progress reports on remediation per- 
formance and subsequent performance in the vocational train- 
ing area were to be maintained. Unfortunately, the project 
began after the beginning of the school year. Skills Checks 
were administered three months after classes began, making 
an over-time study impossible, there was no realistic 
method available to assess the relationship between remedia- 
tion performance and subsequent performance in the voca- 
tional training area. However, on the basis of pre-test 
results, six students in Office Occupations and twelve 
students in Automotive Mechanics were identified as having 
specific deficiencies and were given remediation. At this 
point in time feedback is not available to adequately assess 
results . 
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Since the project has been refunded, the project staff 
has made preparations to meet the above objectives as 
originally outlined. Applicants to Office Occupations, 
Automotive Mechanics, and Machine Shop, are all being 
tested as part of the application procedure. At this 
time, twelve Office Occupations applicants have been 
identified as having skill deficiencies and are currently 
involved in remediation. Case studies, progress reports 
on remediation and subsequent performance in the vocational 
training area will all be included in the June 1976 final 
report. 

10. After all Skills Checks had been field tested and remedial 
education implemented, the Skills Checks were thoroughly 
reviewed and analyzed by the project staff and the voca- 
tional instructional staff. Appropriate revisions were 
made and final forms of the Skills Checks were prepared. 

11, A final report will be prepared. 
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SELECTION OF VOCATIONAL AREAS 



The three vocational training service areas which were selected 
as subjects for this project were: Office Occupations, Auto- 
motive Mechanics and Machine Shop. All three vocational areas 
met project criterion of commonality to a majority of the voca- 
tional institutes and community colleges in the State of Wash- 
ington. 
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DEVELOPMENT OF SKILLS LIST 



Upon identification of each vocational area for test develop- 
ment, the following procedure was utilized in the development 
of the skills tests. 

The determination and development of a pre-entry skills list 
requires three basic steps. First, the researcher performs a 
content analysis of all textbooks used in the selected program. 
The content analysis allows the researcher to become familiar 
with course requirements and expectancy levels regarding skills 
a student should have prior to entry into the course. Expect- 
ancy levels are established by noting where the coursework 
begins. For instance, if math begins with fractions there is 
an underlying assumption that the student is proficient in basic 
math skills of .addition, subtraction, multiplication and divi- 
sion. Also, there is probably an assumption of student famili- 
arity with fractions, at least some knowledge as to what frac- 
tions are and their use. Readabilities are also performed on 
the textbooks to determine the level an entering student should 
be reading. 

Step two requires interviews with all instructors involved with 
the selected program. The project's objectives are briefly 
explained to each instructor, especially project needs for pre- 
entry skills only. All interviews center , around the question... 
What do you believe are the necessary basic skills for a student 
to have prior to entering into your program? It is believed 
open-ended, unstructured interviews tend to result in maximiz- 
ing information and in establishing rapport with instructors. 
On the basis of instructor interviews a tenative pre-entry skills 
list is drawn up. 

Step three requires interviews with several students enrolled 
in the selected program. Students are chosen on the basis of 
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length, of time in the program and grades. It is assumed that 
students with greater length of time in the program will be 
the ones most knowledgeable and opinionated as to what pre- 
entry skills are necessary. It is further assumed that all 
grade levels should be represented so that a range of experi- 
ences can be tapped. '*A" and "B" type students are most likely 
to have sufficient preparatory background and to be able to 
relate what skills helped them the most. 

Other students selected will have a deficient preparatory 
background and, on the basis of the problems they have had 
in class, be able to reveal what they consider necessary pre- 
entry skills. After developing three skill checks, these as- 
sumptions have proved to be warranted. A skills list is devel- 
oped from data obtained from student interviews. 

After all three steps have been completed, a comparision of the 
researcher's familiarity of course textbooks, skills list de- 
veloped from instructors • interviews, and skills list developed 
from student interviews is made. This comparison acts as a 
check and balance system, for inconsistencies are easily no- 
ticed and demand explanation before a final list can be de- . 
veloped. 

It has been our experience that most inconsistencies were due 
to instructors listing skills "taught in class" as desirable 
pre-entry skills. Despite repeated explanation of project ob- 
jectives, several instructors interpreted the skills check as 
a "screening device". Renton Vocational Technical Institute 
sponsors an open-door policy. Due to this policy there are 
very few means available to the instructor to keep out un- 
desirable applicants. If an avenue of rejection is taken by 
an instructor, he is often called upon to defend his action. 
The overall result is that many instructors are forced to ad- 
mit some applicants into their class that they do not want. 
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By listing certain skills as pre-entry, i.e., listing alegebra 
or geometry as pre-entry skills when the course begins with 
fractions, the instructor is putting some **teeth** into the 
skills check which could restrict or discourage admission. 
It Is contended that, in most cases, skills taught by the in- 
structor in the classroom are not legitimate pre-entry skills. 
The skills check is not intentionally designed as a screening 
device. 

After all variation of listed skills has been explained, a 
final skills list is drawn up. 
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OFFICE OCCUPATIONS SKILLS LIST 

The Office Occupations skills list was derived from a content 
analysis of text books, unstructured interviews with three in- 
structors and twelve students, and the Coordinator of Office 
Occupations. Instructors interviewed taught English, math, 
accounting and typing. Students interviewed represented a 
range from poor to excellent students. All instructor inter- 
views centered around the question. .. .What do you believe are 
the necessary skills for a student to have prior to entering 
the Office Occupation's program? Students were asked the same 
question. In addition they were also asked... What problems 
did you have or perceive others to have in Office Occupation's 
coursework? 
Content Analysis 

A content analysis of Office Occupations' textbooks revealed the 
necessity for an applicant to haye skills in English and math. 
In order to do the coursework, knowledge of parts of speech, 
parts of a sentence, capitalization, and punctuation are neces- 
sary. An applicant must be able to perform basic calculations 
of addition, subtraction, division and multiplication of whole 
numbers, and work with fractions, decimals, and percentage. 
Readabilities suggest an applicant's reading level should be 
around tenth grade. It was learned that most of the above 
skills are taught in the classroom. Normally, these skills 
would not meet project needs because they fall in the category 
"skills taught in class". However, after interviewing the 
four Office Occupations' instructors it was determined that it 
would be reasonable to request applicants to have at least 
minimum proficiency in these skills. 

Instructors interviewed argued that although they teach basic 
skills in English and math these skills should be pre-entry 
skills because of two factors. First, since most entering 
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students do have some background in English and math, the in- 



introluctory courses are one semester in length. Combining the 
underlying assumption of previous experience and the time fac- 
tor, one observes the consequence of some students falling be- 
hind, becoming defined as poor students and ending up either 
taking the course over again or dropping out. Instructors an- 
ticipate that if students entered office occupations with at 
least a minimal skill base the chances of failure or frustration 
would be significantly reduced. This logic was adopted in de- 
veloping the final Office Occupations skills list. 



troductory courses are taught with the underlying assumption 
that English and math skills are not new to students. Second, 
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INSTRUCTOR INTERVIEWS 



Instructor I teaches English. She mentioned parts of speech, 
sentence construction and vocabulary as skills most problema- 
tic to students. Sentence construction is defined as parts of 
a sentence, i.e., subject, predicate and direct object. Other 
skills mentioned were capitalization, punctuation, word usage, 
spelling and reading. Vocabulary, spelling and reading skills 
should be at least at the ninth grade level. The instructor 
believes that if students enter Office Occupations with a skill 
base composed of the above they will have minimal difficulty. 

Instructor II te,aches accounting. He is looking for students 
with average intelligence which he defines as sophomore for 
older students and senior for younger. He wants people who 
like to work with figures. It is necessary to have basic 
skills of addition, subtraction, division and multiplication. 
Several students have been having problems with coursework - 
which he relates to insufficient math background, classwork 
overload, not knowing how to study, and don't give a damn 
attitude. His attendance record was marked with several ab- 
sences and tardies. 

Instructor III teaches math and typing. Much of the interview 
was an echo of the previous two. She especially emphasized 
students having problems with decimal placement after addition, 
multiplication and division of decimals. "They simply don't 
know what to do with the decimal." Also, students must know 
fraction to decimal equivalents, percentage and percentage to 
decimal equivalent and decimal to percentage equivalent. 

On the basis of instructor interviews the following list was 
constructed : 

Identifying Parts of Speech 

1. Nouns 

2. Pronouns 

3. Adjectives 



4. Verbs 

5 . Adverbs 

6. Prepositions 

7. Conjunctions 

, Capitalization 

8. First word of every sentence 

9. Brand names 

10. Building names 

11. Business firms 

12. Days, months 

13. Historical events, periods, documents 

14. Institutions 

15. People 

16. School terms 

17. Title of publications 

18. Places 

Identifying Parts of Sentence 

19. Subject 

20. Object 

21. Predicate 

Punctuation 

22. Quotation marks 

23. Apostrophe 

24. Period 

25. Question mark 

26. Exclamation point 

Word Usage 

27. Correct use of pronouns 

28. Similar sounding words 

29. Correct use of verbs 

Spelling 

30. Common words (20 words) 

Vocabulary: Meaning of words 

31. Approximate meaning (synonym) (25 words) 

Reading 

32. Comprehension 
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STUD ENT INTERV IEWS 

Student One: An older student (about 40-45 years of age), 
a very successful worker and ranked as an "A" student by the 
instructor. She has not had any problem with coursework but 
she did mention that punctuation did seem to be a problem with 
some of her peers, especially the use of the comma. Also, 
many students have problems with percentages, especially frac- 
tions to percentages since the machines they use will not make 
the conversion. 

Student Two: Rated as a poor student by the instructor. She 
was a very cautious student, just becoming aware of her poten- 
tial. All mathematical computations, percentages, decimals, 
parts of speech, use of comma and periods were problematic. 
The program has been extremely difficult for her. She wants 
to stay in the program but without remedial training it is 
unlikely she will succeed. 

Students Three and Four were interviewed together. Both 
students were rated as slightly above average. They believe 
ability to add, subtract, divide and multiply are necessary 
skills for a student to have prior to entry and they have 
noticed that several students have problems with parts of 
speech 

Student Five: Rated as a "C" student. She was very reclusive, 
difficult to interview, but she did mention that she has had 
problems with percentages and converting fractions to deci- 
mals. She was not aware of any problems other students were 
having. 

Student Six: A foreign student who was rated high in math 
but low in English. He stated that he had no problems with 
math, but English (not his native language) was a serious 
problem. Vocabulary, spelling and punctuation were his main 
problems. Basic calculations, fractions, decimals and percents 
were mentioned as necessary pre-entry skills. 
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vStudent Seven: Rated as "A" by the instructor. She had no 
problems herself but believed other students were having 
troubles with fractions, decimals and percentages. At this 
point she had not taken the English class and therefore was 
not familiar with particular English problems. 

Student Eight: A "C" student who was unfamiliar with prob- 
lems in English. Fractions, decimals and percentages were 
pointed out as problem areas. A problem that has frustrated 
her as well as others is that students are expected to learn 
how to use a machine with minimal instructions and mathemati- 
cal processes at the same time. Many students have not used 
math for several years and being unfamiliar with adding 
machines makes it unnecessarily difficult. A math review prior 
to coursework would benefit many students. 

Student Nine: A "C" student who was a beautician. . She be- 
came discouraged with low pay, long hours and weekend work, 
and is now training for an office job. She has been away 

from school for 18 years. has no problems with math but 

English is rather difficult. There is a definite need for 
basic mathematical skills and knowledge of fractions, deci- 
mals and percentages. Parts of speech is her problem area. 

Student Ten: A "D" student originally from Alaska where she 
received inadequate training in math and English. She had 
enrolled in business occupation's English course, but was 
forced to drop the course after it proved to be difficult. 
On her own initiative she is taking an English review course. 
Other than a locator test she was not tested to see whether 
she had the skills to successfully participate in Office oc- 
cupations coursework. 
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Student Eleven: An older student rated •'C* who once was a 
beautician and is now training for another job. She has 
experienced difficulty with math and has taken self-review 
courses. Basic computations are a weakness, especially multi- 
plication. She has had no problems with English. 

Student Twelve: A college student ranked "A", taking classes 
in Office Occupations. She believed all basic math and Eng- 
lish skills were necessary for a student to have prior to entry. 
Four of her friends dropped out of RVTI^s summer courses due 
to lack of appropriate skills necessary for course assignments. 
They also had become frustrated trying to learn how to operate 
adding machines while not fully understanding mathematical 
procedures . 

At completion of student interviews a skills list was drawn up 
and compared with the skills list derived from instructor inter- 
views. No inconsistencies were noted. However, an addition 
was necessary. Students mentioned the "use of comma" as a neces- 
sary prerequisite whereas instructors did not. Instructors were 
asked if "use of comma" was a justifiable pre-entry skill. They 
supported the skill and it was incorporated into the final list. 

After the content analysis, interviews, and resulting skills 
list were completed, the Director of Office Occupations was re- 
quested to review and comment on our efforts. She was satis- 
fied that the skills list included all pre-entry skills perti- 
nent to Office Occupations. She was very supportive in all 
ways with the exception of some reservation about cover design. 
This will be worked out later. 

The final skills list was the same as the one developed from 
instructor's interviews with the addition of the "Use of Comma" 
skill: 

a) Separate words and numbers in a series 

b) Set off an appositive 



c) After a dependent clause at the beginning of a sentence 

d) Before quotations 

e) With addresses and dates 

f) Set off such words as "of course, indeed, for instance, 
moreover, no doubt" 

g) After introductory words which are separated from the 
rest of the sentence 

h) Before a coordinating conjunction in a compound sentence 
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AUTOMOTIVE MECHANICS SKILLS LIST 



The Automotive Mechanics skills list was derived from .a content 
analysis of course textbooks, interviews with the three auto- 
motive instructors, and interviews with twelve students. Stu- 
dents were selected by the head instructor on the basis of 
"grades" and "length of time in the program". 

Content Analysis 

A content analysis of automotive mechanics' textbooks revealed 
that a student should have a working knowledge of basic math 
skills, fractions, decimals and some percentage. Readabilities 
rated textbook reading difficulty between 10.2 - 12.5. It was 
also noted that a student will be confronted with many new 
words. Without a mechanical background, vocabulary may prove 
to be a problem. Since these skills are not taught in the class- 
room, it is believed to be especially important for applicants 
to have some competency in these areas before entering the auto- 
motive program. 

Instructor Interviews 

There are three automotive mechanics instructors. All three 
were interviewed at the same time. The project's objectives 
were explained repeatedly because it was necessary to make 
sure the instructors understood that our project was not con- 
cerned with developing a screening device. Instructors were 
not satisfied with the caliber of several students and wanted 
a means of eliminating them or screening them out. After 
reaching an understanding, the instructors appeared very sup- 
portive. Some caution should still be exercised as to how 
they perceive the use of the proposed skills check. Their 
initial interpretation is excellent reason why skill check 
results should be interpreted and used by an objective or 
neutral party. 
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All three instructors agreed upon the following skills as 
necessary pre-entry skills: 
Basic computations 

1. Addition of whole numbers 

2. Addition of money: dollar sign and decimal point 

3. Subtraction of whole numbers 

4. Subtraction of money using dollar sign and decimal 

5. Multiplication of whole numbers 

6. Multiplication of money using dollar sign and decimal 
point 

7. Division of whole numbers 

8. Division of money using dollar sign and decimal point 

9. Computing averages 

Fractions 

10. Addition of fractions 

11. Subtraction of fractions 

12. Multiplication of fractions 

13. Division of fractions 

14. Reducing fractions 
15* Unlike fractions 

16. Fractions to decimals 

17. Fractions to percent 

Decimals 

18. Reading and writing decimals as numbers and words 

19. Addition of decimals 

20. Subtraction of decimals 

21. Multiplication of decimals 

22. Division of decimals 

23. Comparing decimals 

24. Rounding off 

25. Decimal to percent 

Percentage 

26. Percent to decimal 

27. Percent of a total 

Reading 

28. Comprehension (with grade level) 
Vocabulary 

29. Approximate meaning (synonym) 
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Student Interviews 



Student One: A "B" student who has been in the program since 
September 1974 mentioned basic computations, precentages, read- 
ing and comprehension as very important pre-entry skills. Read- 
ing was most important with desired reading level placed at 
twelfth grade. This student supported the need for a test be- 
cause of so many students having trouble with shop and class- 
room problems. 

Student Two: This student has been in the program for 1 1/2 
years and is rated "one of the best" by instructors. Basic 
calculations, fractions, decimals, percentages, vocabulary and 
reading were all mentioned as important pre-entry skills. He 
also supported our project *s objective. 

Student Three: Student has been in the program for two years 
and is also rated "one of the best". His responses were the 
same as Student Two's. Basic computations, fractions, decimals, 
percentages, vocabulary and reading were all listed as necessary 
pre-entry skills. 

Student Four: Ranked "C" by instructors and has been in the 
program since August 1974. Again, basic computations, frac- 
tions, decimals, percentage, vocabulary and reading were all 
given as important pre-entry skills. 

The remaining interviews, regardless of "length of time in pro- 
gram" and "grades" proved to be a repeat of skills already noted. 
Students interviewed see a real need for a skills check instru- 
ment and were very supportive toward the project. This attitude 
is most likely due to the fact that academic skills necessary to 
pass coursework and solve shop problems are not taught in class. 
It is assumed that students entering Automotive M echanics will 
have the necessary skills. Apparently, this is not the case. 
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There were no inconsistencies or additions to the skills list 
supplied by the instructors. The content analysis was also 
compatible with this list. Therefore, the skills list developed 
from instructor interviews is the final form of pre-entry skills 
for Automotive Mechanics. 
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MACHINE SHOP SKILLS LIST 



The Machine Shop skills list was derived from a content analysis 
of course textbooks, interview with the instructor, and inter- 
views with twelve students. Students were working on various 
projects at the time of interviewing and as a result several 
students were not present. Therefore "availability" was the 
determining factor as to who was interviewed. Although "length 
of time in the program" was not a deciding factor, a good spread 
was obtained. "Length of time in program" for interviewed 
students was.... since January, 1975 = three students; since 
September, 1974 = three students; since June, 1974 = two stu- 
dents; for more than one year = four students. Grades ranged 
from ''C' to "A". 

Content Analysis 

A content analysis of Machine Shop textbooks suggests a need 
for basic computations, fractions, decimals and about twelfth 
grade reading ability. Since the entering student will be con- 
fronted with machine shop jargon in the classroom and shop, 
vocabulary may be a necessary pre-entry skill. 

Instructor Interview 

The interview began the same way others had. The instructor 
desired a screening device to turn away certain students. Re- 
peated explanation of project's objectives was necessary. After 
an understanding was reached the instructor listed the follow- 
ing skills as desirable pre-entry skills. Basic computations 
as well as some knowledge of fractions and decimals are a must. 
The instructor does teach math beginning with fractions and 
ending with algebra. Normally, our purposes would suggest that 
since fractions and decimals are taught in class we should not 
use them as pre-entry skills. However, in this case it is 



believed necessa.ry to have a check on them. (It was learned 
that math instructions moves rapidly and is not extensive. 
Besides the time factor, most students when interviewed, vol- 
untarily expressed a low opinion of the instructor's ability 
to adequately teach math.) 

Reading and vocabulary were cited as necessary prerequisites 
by the instructor. One of the problems facing the instructor 
is the time he wastes by telling or showing a student over and 
over again what something is or how to find it. He explained 
that if a student could read and comprehend and had a good 
vocabulary they could assist themselves and release him for 
other duties. 

He also mentioned he would like a means by which an applicant's 
logic could be checked. Further probing revealed that by check- 
ing an applicant's logic, he meant a check on an applicant's 
ability to Conceptualize, i.e., identify an object from three 
different viewpoints. This skill may be outside project's ob- 
jectives. The skill is taught in the class under the label 
"Blueprint Reading". 

In review, skills listed as important pre-entry skills by the 
instructor are: 

Basic Calculations 

1. Addition of whole numbers 

2. Subtraction of whole numbers 

3. Multiplication of whole numbers 

4. Division of whole numbers 

Fractions 

5. Addition of fractions 

6. Subtraction of fractions 

7. Multiplication of fractions 

8. Division of fractions 

9. Fractions to decimals 
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Decimals 

10. Addition of decimals 

11. Subtraction of decimals 

12. Multipliciition of decimals 

13. Division of decimals 

14. Decimals to fractions 

15. Comparing decimals 

16. Rounding off 

Reading 

17. Comprehension 
Vocabulary 

18. Approximate meaning (synonym) 

Conceptualization 

19. Three viewpoints 

Student Interviews 

Student One... Student was rated "C" and has been in the pro- 
gram since September, 1974. Pre-entry skills mentioned as im- 
portant were l)asic computations, fractions, decimals. He 
thought the skills check was a good idea and believed previous 
machine shop experience would be a good addition to the check. 

Student Two... Student was ranked "C" and had been in the pro- 
gram since September, 1974. Reading, ' vocabulary , basic com- 
putations, fractions, and decimals, manual skills and a visual 
test on dimensions were listed as necessary pre-entry skills. 

Student Three... Student was ranked "B" and had been in the 
program since June, 1974. He mentioned basic computations, 
fractions, decimals, algebra, geometry and a visual test on 
dimensions as necessary pre-requisites . 

Student Four... Ranked "A" by instructor and has. been in the 
program since September, 1974. She was experiencing no diffi- 
culty with coursework, in fact, she was tutoring some of the 
students. She did mention that she was finding it difficult 
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to be accepted in a trade that had traditionally attracted 
only males • Skills listed as necessary pre-entry skills were 
basic computations, fractions, decimals, reading, vocabulary 
and a dimensions test* 

After the fourth interview, interviews reached a plateau of 
similarity; No additions or inconsistencies between the first 
four interviews and the remaining eight were noted. The skills 
list was made up of the following skills: basic calculations, 
fractions, decimals and reading. Vocabulary was believed neces 
sary by five students. Seven students did not list vocabulary 
as important. At first glance this may appear to show lack of 
support for having a vocabulary skill. It was thought that 
by noting a student's length of time in the program, an explana 
tion may be offered. Students who listed vocabulary as impor- 
tant may have spent less time in the program and therefore had 
some difficulty with new words, whereas older students should 
have incorporated most shop orientated words into their vocab- 
ulary. This was not the case. The seven students who did not 
list vocabulary as important represented all points along the 
"length of time in program" dimension. So did the five who 
listed vocabulary as important. Perhaps this is an illustra- 
tion of inconsistency, but there appears to be enough support 
to include vocabulary in the skills list, 

A check to determine an applicant's ability to conceptualize 
was mentioned as important by six students. Combined with in- 
structors expressed desire for a check on this ability supplies 
enough support for its inclusion in the final check list. How- 
ever, it must be determined if conceptualization falls within 
the project's domain, A decision will have to be made before 
conceptualization is included in the skills list. 

Noting no additions or significant inconsistencies the -Pin^^l 
skills check will be the same as the one developed from data 
obtained from the instructor's interview. 



SKILLS LIST 



The following skills lists are in final form for Office Occupa 
tions, Automotive Mechanics and Machine Shop: 

Office Occupations 

Identifying Parts of Speech 

1. Nouns 

2. Pronouns 

3. Adjectives 

4. Verbs 

5. Adverbs 

6. Prepositions 

7. Conjunctions 



Capitalization 



8. 


First word of every sentence 


9. 


Brand names 


10. 


Building names 


11. 


Business firms 


12. 


Days, months 


13. 


Historical events, periods, documents 


14. 


Institutions 


15. 


People 


16. 


School terms 


17. 


Title of publications 


18. 


Places 



Identifying Parts of Sentence 

19. Subject 

20. Object 

21. Predicate 



Punctuation 

22. Quotation marks 

23. Apostrophe 

24. Period 

25. Question mark 

26. Exclamation point 

Word Usage 

27. Correct use of pronouns 

28. Similar sounding words 

29. Correct use of verbs 

Spelling 

30. Common words (20 words) 
Vocabulary: Meaning of words 

31. Approximate meaning (synonym) (25 words) 



Reading 
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Automotive Mechanics 



Basic computations 

1. Addition of whole numbers 

2. Addition of money: dollar sign and decimal point 

3. Subtraction of whole numbers 

4. Subtraction of money using dollar sign and decimal 

5. Multiplication of whole numbers 

6. Multiplication of money using dollar sign and decimal point 

7. Division of whole numbers 

8. Division of money using dollar sign and decimal point 
9* Computing averages 

Fractions 

10. Addition of fractions 

11. Subtraction of fractions 

12. Multiplication of fractions 

13. Division of fractions 

14. Reducing fractions 

15. Unlike fractions 

16. Fractions to decimals 

17. Fractions to percent 

Decimals 

18. Reading and writing decimals as numbers and words 

19. Addition of decimals 

20. Subtraction of decimals 

21. Multiplication of decimals 

22. Division of decimals 

23. Comparing decimals 

24. Rounding off 

25. Decimal to percent 

Percentage 

26. Percent to decimal 

27. Percent of a total 

Reading 

28. Comprehension (with grade level) 
Vocabulary 

29. Approximate meaning (synonym) 
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Machine Shop 



Basic Calculations 

1. Addition of whole numbers 

2. Subtraction of whole numbers 

3. Multiplication of whole numbers 

4. Division of whole numbers 

Fractions 

5. Addition of fractions 

6. Subtraction of fractions 

7. Multiplication of fractions 

8. Division of fractions 

9. Fractions to decimals 

Decimals 

10. Addition of decimals 

11. Subtraction of decimals 

12. Multiplication of decimals 

13. Division of decimals 

14. Decimals to fractions 

15. Comparing decimals 

16. Rounding off 

Reading 

17. Comprehension 
Vocabulary 

18. Approximate meaning (synonym) 
Conceptualization 

19. Three viewpoints 
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CONSTRUCTION OF TEST ITEMS 



Before item construction begins it is necessary to resolve 
certain issues. It was pointed out in the Statement of Prob- 
lem section that standardized tests used for diagnosing skills 
or predicting success of vocational school applicants have fal- 
len short in two areas. Many standardized tests contain items 
that are not relevant to the applicant's chosen program. Also, 
standardized tests often have items that are culturally biased 
or rely too much-upon verbal skills. One must ask what is 
being tested... an applicant's socio-economic status, ability 
to read and communicate, or vocationally-related skills? It 
seems most inappropriate to make decisions about an applicant's 
competencies and probability of success in his/her chosen vo- 
cational area upon results influenced by items tapping skills 
that lie outside of that area. Thus, items for vocational 
tests should be designed to circumvent the problem of incorpor- 
ating extraneous factors into test results* 

Another problem that must be dealt with is the assumption of 
validity that one makes when developing an item. That is, the 
developer assumes that the item will measure what he intends 
it to measure. To assure valid test items, underlying assump- 
tions must be put to task. For this purpose, as well as others, 
the contractual services of Dr. Charles Schultz were relied 
upon.^ The value of his objective and competent eye cannot be 
over -estimated. 

It is believed that the following procedure used for developing 
test items resolves the problems listed above. Initially, item 
construction begins with the knowledge and judgment of the 
assistant researcher. The content analysis and interviews used 
to develop the skills list serve as an excellent springboard 



2. The professional opinions and judgments of Dr. Charles 
Schultz, Third-Party Evaluator, were utilized throughout all 
phases of item construction. 



for construction of test items. If the research assistant is 
thorough, he will be familiar with each program's social and 
physical settings, equipment and related problems. This in- 
formation forms an excellent resource pool which can be drawn 
upon for item construction. Items can be designed to tap a 
skill while framed within the context of a particular vocational 
program. So, not only are the skills specific to a particular 
program, but the way they are presented is specific as well. 

All vocabulary words used in a Skills Check met two criteria. 
They are common to the program that the Skills Check is designed 
for, and they are predicted to be at the ninth grade 1-evel of 
difficulty. 

The Reading Comprehension section of a Skills Check is com- 
prised of what is considered to be neutral reading material. 
That is, the material is such that there is not much chance 
that ''previous knowledge" of the applicant will be a factor. 
Often material used to measure reading comprehension is techni- 
cal in nature, i.e., a passage on how to operate a certain 
machine. This often results in giving those who have worked 
with such a machine, or are familiar with whatever the passage 
is about, an unfair advantage. It is quite possible that such 
a reading comprehension check will measure an applicant's pre- 
vious experience rather than reading comprehension. Neutral 
material was considered to be historical in content. The 
Office Occupation's Reading Comprehension section was made up 
of three sections titled "History of the Typewriter," "History 
of Typewriting Instruction," and "History of Typewriting Text- 
books." It is believed the chances of applicants being famil- 
iar with this material is minimal. Given pre-test and current 
testing results, this assumption appears to be reasonable. All 
reading comprehension sections were designed to meet the read- 
ability level required by the program. 
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All items designed to tap English and math skills were express- 
ed in a way specific to a particular program. For instance, 
an addition problem appearing in a Machine Shop Skills Check 
could require the applicant to sum tolerances of a drive shaft. 
The same skill presented in an Office Occupations Skills Check 
could require the applicant to sum all the sales made in a day 
by a large business firm. In Auto Mechanics the same skill 
could be expressed by requiring the applicant to add up a 
service bill. 

Another issue that must be dealt with is, how many items are 
sufficient to measure a skill? In math sections of the Skills 
Checks it was observed that some skills, such as selecting the 
largest decimal from a group of five, required only one opera- 
tion (comparing decimals), and could be adequately measured by 
one or two items. A skill that requires more than one opera- 
tion for completion, however, should be measured by three or 
four items. A story problem in addition requires the appli- 
cant ,to read, through the problem, select pertinent figures, 
transfer them to provided working space and list them hori- 
zontally. This procedure could easily cause errors that result 
in an incorrect answer, independent of the addition skill being 
measured. Additional items tend to compensate for fortuitous 
errors and offer a more valid assessment o£ an applicant's 
skills. Therefore, the number of items believed necessary to 
adequately measure a skill were determined by the probability 
of fortuitous error factor. Pre-tests have given further cre- 
dence and direction to this procedure. 

The number of items measuring Spelling, Vocabulary, and Read- 
ing were determined by noting the number of items other such 
tests relied upon. In addition, spelling items were selected 
according to how well they covered the general rules of spel- 
ling. 
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Items measuring English skills, specifically Parts of Speech, 
Capitalization, Parts of a Sentence, Punctuation and Word 
Usage need to cover several usages of a/skill. In Punctuation 
there are eight different situations that require the use of 
a comma, eleven different situations that require capitaliza- 
tion. Items must cover all situations applicable to a speci- 
fic skill in order to adequately measure the applicant's pro- 
ficiency with that skill. Therefore, the number of items 
designed to measure the above listed skills was determined by 
what was required to measure all uses of the skills. 
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REVIEW AND CRTTIOUF. OF TKST ITEMS 



After the Research Assistant had completed development o£ 
criterion-referenced items for each skills list, a meeting was 
scheduled between the Principal Investigator, Research Assis- 
tant and Third-Pary Evaluator. At this meeting, all items were 
evaluated by the Third-Party Evaluator and maintained or revised 
according to how well they met conditions of: 1) Clarity of 
directions. Each item must be explicitly clear regarding what 
it requires the applicant to do. 2) Validity of test items. 
There must be an. ex act -fit between the item and skill. Anything 
less could result in the item measuring something other than 
what was intended. 3) Level of difficulty. Items must be at 
the determined level of difficulty. Most items for the three 
Skills Checks are at the ninth grade level of difficulty. 4) 
Number of items necessary to measure a skill Calready discussed) 
5) Relationship between item and skill. Items were reviewed 
as to how well they met requirement of being specifically de- 
signed for a particular program. 
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PRE-TESTS 



Upon completion of item revision, pre-test arrangements were 
made. The pre-tests served the dual function of identifying 
those students with specific weaknesses and providing informa- 
tion needed to determine what items were suitable to project 
needs and what items needed revision (discussed in Item Analy- 
sis section) . 

The Skills Checks were administered to 90 students in Office 
Occupations, 71 students in Automotive Mechanics and 50 students 
in Machine Shop. 
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MEASURES OF VALIDITY 



A validity check is essentially a procedure designed to measure 
the effectiveness of a testing or measurement instrument. It 
is a comparison between what authors say their instrument will 
do (theory) and what the instrument actually does do (reality) . 
If there is a close fit between theory and reality then the 
instrument, for all intents and purposes, is valid. That is, 
the instrument does what it was designed to do. If there is 
not a tight fit, then the instrument is measuring something 
other than what it was designed to measure and revision is 
necessary. 
Office Occupations 

In order to determine if the Office Occupations Skills Check 
was discriminating between proficient and deficient students, 
a comparison was made between students "new" to Office Occupa- 
tions and successful students "old" to Office Occupations. 
"New" students were defined as those students who were begin- 
ning coursework or had not completed one full semester at date 
of test. "Old" students were defined as those students who 
had completed one or more semesters of coursework with at 
least a "C" average at date of test. 

Since many of the pre-entry skills that appear on the Office 
Occupations Skills Check are reviewed in class, it was assumed 
that "successful old" students, having gone through at least 
one semester of coursework, should score higher than "new" 
students. As a group, the "successful old" student mean (X) 
should be higher than the "new" student mean. No difference 
between means could indicate one or more of the following cir- 
cumstances: 1) The items are not discriminating at an appro- 
priate level; 2) Level of difficulty needs revision; 3) The 
unlikely possibility that "new" students are as proficient 
with pre-entry skills as are "successful old" students; 
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4) The skill is not a suitable candidate for testing in Office 
Occupations; 5) Instruction needs revision. 

Graphs 1 thro IX all reveal mean differences in the expected 
direction. Differences in mean scores for skills in Math, Parts 
of Speech, Punctuation and Spelling appear to be significantly 
different. 

An external validity measure was implemented to determine the 
predictive ability of the Office Occupations Skills Check. At 
the end of the semester, the English scores for each "new'* 

3 

student were compared with the English instructor's evaluation. 
Evaluation was accomplished by "Q-Sorf* method. The instructor 
was handed cards with a "new'* student's name on each card. The 
instructor was then requested to arrange the cards according to 
student proficiency. The students were ranked from most profi- 
cient to least proficient student. Attendance and motivation 
were not considered. The instructor directed herself to only 
questions of, can the student do the work and how well can 
he/she do it? If the instrument has good predictive ability then 
one should observe a high correlation between Skills Check Eng- 
lish scores and teacher's evaluation. Those "new" students who 
scored higher on the English part of the Skills Check should 
also be evaluated higher by the English instructor. 

Graph X indicates that the Office Occupations Skills Check has 
reasonably good predictive ability regarding English skills. 



3. At this point in time, "new" students had not taken the Of- 
fice Occupation's math course. Therefore, an external validity 
check on math skills was not possible. However, since the pro- 
ject has been refunded, this information will be in the June, 
1976 report. 
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Automotive Mechanics 



Unlike instructors in Office Occupations, Automotive Mechanics' 
instructors teach numerical -verbal skills as required by the 
trade. Since instructors do not emphasize these skills, stu- 
dents are graded on other factors such as motivation, atten- 
dance, and work assignments in the shop and classroom. As a 
result it was not possible to compare "new'' students with 
"successful bid" students. Students defined by the instructor's 
evaluation as "succesful old" students were successful in ways 
other than numerical -verbal proficiency. This is illustrated 
by the finding that ten out of twenty-three "successful old" 
students, rated high (B grade or better) by instructors, received 
scores in the math section of the Skills Check that were no- 
ticeably lower than the group mean. Further, a Q-Sort techni- 
que revealed that although Skills Check scores for "new stu- 
dents" appear to be in the right direction, there are several 
negative cases. (See Graphs XI, XII, and XIII) 

In light of this, the question arises, if numerical -verbal 
abilities are not important for success, why teach them? In 
reponse to this question, the content analysis, student and 
instructor interviews supported the need for an entering 
student to have numerical-verbal skills. This suggests that 
perhaps instructional materials should include numerical -verbal 
skills and that evaluation of students should be expanded. 

At this point in time the Automotive Mechanics Skills Check 
appears to be meeting project needs > i.e., identifying those 
students who exhibit deficiences. Graphs XIV, XV, and XVI 
illustrate recorded scores of "new" students. It can be ob- 
served that some students scored noticeably lower than others. 
These students were located for remediation. 
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Mac hine Shop 

At the time of testing there were no "new" students in the 
Machine Shop Pro}»ram. Forty-three "old" students were tested 
and their test results were graphed. These graphs will serve 
as referees regarding issues of discrimination between profici- 
ent and deficient students. Like Office Occupations, many of 
the pre-entry skills that appear on the Machine Shop Skills 
Check arc reviewed in class. Therefore, a comparison of means 
between "old" students who have undergone machine shop training 
and "new" students who have not undergone machine shop training, 
should reveal that "old" students, as a group, score noticeably 
higher than "new" students. 

After Fall semester, a Q-Sort will be administered to th^»' in- 
structor. The instructor will be requested to evaluate "new" 
students after one semester of training. Evaluations will be 
compared with Skills Check scores obtained by entering students 
at the beginning of Fall semester. If the Skills Check has 
predictive value there should be a high correlation between in- 
structor's evaluation and student's score. 

Graphs illustrate the numerical -verbal placement of "old" 
students and the means (X) that will be used in comparison with 
"new" students' scores. 
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ITEM ANALYSIS 



Besides dealing with issues of validity, information gained 
through pre-testing was used for performing item analysis. 
All items used in pre-tests underwent item analysis* Item 
analysis is a non-complicated statistical technique specifi- 
cally designed to assess the discriminatory power of a test 
item. By discriminatory power it is meant the ability of an 
item to discriminate between students with proficient skills 
and students with deficient skills. Ideally, every item in 
final form would have discriminatory power that would allow 
proficient students to be the ones most likely to get the item 
correct and deficient students to be the ones most likely to 
miss the item. Since the purpose of this project is to make 
visible students who need remedial education, given vocational 
interests, item analysis speaks directly to project needs. 

Item Analysis Procedure : 

After the exam has been administered and corrected, each section 
composed of X number of items is divided into groups of Hi or 
Lo. It is necessary for each group to have the same amount of 
students. The best way to proceed is to place all scores for a 
particular section in a range going from lowest to highest score. 
Drop the low and high extremes before you divide into Hi, Lo 
groups. This will create a population that approximates a nor- 
mal distribution. (Normal distributions are more amenable and 
reliable for statistical procedures such as item analysis. See 
''Statistical Reasoning in Sociology,*' Mueller, Schuessler, Cost- 
ner, for discussion to this point). To insure reliable results 
each group should have an N of at least 20. Figure I illustrates 
this initial step. 
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FIGURE I 




1) Drop extremes A, B, and C 

2) Divide remaining scores into two equal groups. 
With this population Hi « 14 - 19, Lo = 8 - 13 
There are 24 in each group. 



Number all of the items that appear in the section. If an 
item has two or more answers number each answer separately, 
i.e., if item one had three answers it would be numbered la), 
lb), and Ic) . Review each exam noting whether the individual 
is in the Hi or Lo group. For each correct answer, place a 
tally mark behind the item^s corresponding number in either 
the Hi or Lo column. Incorrect answers are not tallied so 
the completed tally sheet will be a 'rount of correct answers 
only for both Hi and Lo groups. 

Figure II shows the completed tally sheet for Office Occupa- 
tions, Spelling: 
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FIGURE II 



HI 



LO 



personnel 

Corporation 

intention 

vocational 

review 

statement 

employer 

Business 

recommendation 

flexible 

forty 

twelve 

occupation 

similar 

description 

developing 

procedures 

comparison 

eligible 

truly 



mi 
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mi 
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mi. 


11 


im 


mri 


11 






mi 


mi 
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mi 










mi 


mi 


mi 
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mi 


mi 


mi 


mj. 
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mj 


mi 


mi 


mi 


mi 


mi 


mi 
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m4 


mi 


mi 


mti 


mi 


m^i 


m.1 


1 




mi 


mi. 






11 










mi 


m4 


111 




mi 


m^ 


1 







Go through each item noting how many of the Hi group and Lo group 
got the correct answer. If the item is discriminating there should 
be a noticeable difference between the Hi/Lo populations. With an 
N of 20, if the Hi group scores 5 more correct answers than the Lo 
group, the item is probably satisfactory. Anything less than 5 
suggests the item needs to be made more difficult. If a negative 
relationship is noted, i.e, more of the Lo group got the item 
correct than the Hi group, the item should either be dropped or 
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completely changed. If almost all examinees in both groups mas- 
sed the item then the item may be a good candidate for omission 
because of the difficulty factor. Figure III, a totaled tally 
sheet, illustrates an array of discriminatory and non-discriminatory 
items. 

FIGURE III 

Item Analysis for Spelling, Office Occupations; New 
Entering Students 



Hi = 14 - 19 
Lo = 8-13 


25 in 
Hi 


each 
Lo 


group 


Hi 


Lo 


personnel 


16 


8 


forty 


18 


11 


Corporation 


24 


11 


twelve 


24 


21 


intention 


23 


15 


occupation 


23 


21 


vocational 


23 


21 


similar 


22 


12 


review 


21 


10 


description 


16 


5 


statement 


22 


14 


developing 


24 


23 


employer 


, 24 


14 


procedures 


24 


16 


Business 


23 


21 


comparison 


24 


16 


recommendation 


15 


8 


eligible 


15 


2 


flexible 


18 


2 


truly 


18 


11 



Items 1, 2, 3, 5, 6, 7, 9, 10, 11, 14, 15, 17, 18, 19 and 20 are 
all good discriminators, especially items 2, 5, 7, 10, 14, 15 and 
19. Items 4, 8, 13 and 11 are poor discriminators and should be 
changed. There are no negative items. Item analysis reveals 
that this section needs some revision, but overall is doing an 
excellent job differentiating proficient spellers from deficient 
ones. 

Item analysis should never be the final word in item development. 
It is only a mathematical indicator of how well items are work- 
ing, given your particular objectives. If items are developed to 
tap basic skills such as addition of whole numbers and item analy- 
sis of pre-test results reveal that there is a positive difference 
of only one between Hi/Lo groups, a change is not always necessary, 
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Given the skill involved, a difference of one may be good. 
Since the item represents a basic skill it is expected that 
most examinees will answer the item correctly. If they do not, 
perhaps the item should be considered a basic skill .. .perhaps 
the skills listed as important pre-entry skills are too diffi- 
cult. There are many issues that item analysis does not deal 
with. The test developer must rely upon his own judgment as 
to whether an item is working. Item analysis should be considered 
a judgment aid. Any other use of item analysis could result in 
fitting the data to the statistical model rather than fitting 
the statistical model to the data. 
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DEVELOPMENTAL FLOW CHART 
OF 

PRE-ENTRY CRITERION-REFERENCED TESTS 



Content analysis of 
Text Books 



Interview 
Instructor? 




Interview 




Establish skills list 



Initial development of 
test items 



First critique and evaluation 
OF test items by third party 
evaluatqr 



Revision of certain items 



Pretest 



External validity check. 

(Comparison of test scores 
with instructor's evaluation ) 



^ Item analysis 
of all test items. 



Revision of certain test 
items based on item analysis 
and recommendations of third 
party evaluator. 
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CONCLUSION S AND CONTI NUATI ON 

During the initial grant years, FY 75, Renton Vocational Tech- 
nical Institute developed three instruments for prescribing com- 
pensatory education. During the continuation grant year, Ren- 
ton Vocational Technical Institute intends to develop an addi- 
tional six test instruments. Areas anticipated for test develop- 
ment will be electronics, cashier checking, civil engineering, 
technology, major appliance repair, welding, and food preparation- 
food service. It is anticipated that this test development for- 
mat will continue that developed during the initial grant year. 

Other institutions have indicated interest in the Skills Check 
development project. Therefore, it is anticipated that in the 
forthcoming year, project staff will work cooperatively with 
other institutions in the development of Skills Checks. The 
test technician, based at Renton Vocational Technical Institute, 
will work in close cooperation with designated individuals from 
other institutes. 

It is anticipated that the cooperation with the other institu- 
tions will give this project more generalized applicability 
throughout the state of Washington. 

Following development of the test instruments, all will be 
field tested, both at the institutions in which they are gener- 
ated and in other locations. Following the field tests, care- 
ful item analysis and evaluation of external as well as i'AteV- 
nal sources of documentation wi^l be utilized. - 

Documentation generated through the field test phase of the 
project will be utilized in revision of all tests. The 
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external evaluator. Dr. Charles Schultz, will give close super- 
vision to field tests, test validation, item analysis, and re- 
vision of all tests. 



A final set of test instruments will be prepared for printing. 
These final tests will be prepared in a format for reproduction 
by other institutions. 

A new objective for the continuation project of FY 76 is the 
development of a teacher ^s manual to accompany the specific 
tests. This manual will contain an introduction to all tests 
developed, answers for all developed tests, information on 
proper utilization of test scores for the prescription of re- 
quired compensatory education, and an overview of the method 
of item analysis developed for the purposes of the project. 

Statewide dissemination of research projects always presents 
a problem for vocational educators. It is anticipated that 
in the spring of 1976, this project will be at such a level 
of development that it will be appropriate to present the 
project to state workshops, and to individual schools as 
requested. The project researcher will be available for such 
project dissemination. He will be equipped to go to specific 
locations throughout the state to present information on this 
specific project. 

A final report will be generated in June of 1976. This report 
will contain all tests, all information contained in the 
teacher's manual, and additional information germane to the 
development of the project. In addition, it will contain the 
final report of the project evaluator. Project evaluation 
will be based upon the following: 

A. Number of tests developed. 

B. Number of students evaluated through tests. 

C. Number of students receiving remedial education as a 
result of the tests. 
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Number of students successfully enrolled in vocational 
training programs as a result of the tests. 
Subjective case studies generated by instructors in 
those areas in which tests are being (developed. 
Report generated by the external evaluator. 
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^0MPEV5AT0RY ROU^ATIOI ^OR VCnATION TSCHsIGAL TRAINEES 
— A Third Party '^"valuation — 



Jtudents are somf^times unsuccessful in their classes when they 
lack certain skills that are basic to their coursework. Students 
sometlmesr wait ^or months to get into a technical program and then 
f'ail in the Tsro^^ram becaase of poor educational preparation. The 
Gta^f at Renton Vocational Technical Institute (RVTI) wants to pre- 
vent these failures. 

This project aims to develop a way to identify the need ^or 
Dreparation in the basic skills. When a skills deficit is identic- 
^ied, it can be eliminated before the student begins the program. 

Craig Mueller was selected as the Research Assistant for the 
project and has carried the lar'-^est share of the work. Three courses 

study were selected to represent a cross section of training offered 
at R^m. These are Office Occupations, Automotive Mechanics and 
Machine Shop. 

The first step is to find out what skills are needed. This is 
addressed throu/^h a content analysis o^ course materials and inter- 
views with instructors and studerits. The instructors tended to de-- 
fine the elementary levels of their subject matter as "prerequisites". 
It appears they would screen out the less proficient students and 
teach those who were more advanced. However, the school's policy is 
to accept all students and teach them what they need. 
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While the areas of prerequisite r>kills have been pretty well de- 
'^inedi the level proficiency needed is still open to discussion. 
However, It is possible to develop tests that are sensitive across 
a ran^e of proficiency levels. 

The next stet) was to develop tests for the basic skills identified* 
Mueller carried out the test development. Test content was derived 
lar/3jely from textbooks used for the courses and for more basic cov- 
erafje of the orerequislte skills. Course instructors were called 
UT)on .for advice and review. 

Content of test items was identified so that specific skills 
could be isolated. For example a test item measuring punctuation 
were differentiated so that a skill such as "using a comma to set off 
an appositive" could be isolated. More precise definitions of speci- 
fic skills were possible for writing and mathematics than fr)r vocabu- 
lary and reading comprehension. 

The sta^^ attempted to ascertain the appropriate content and 
difficulty level for test material from analysis of textbooks and 
interviews. Then the tests were tried out on students who varied in 
ability and amount of course work. These test results were analyzed 
to find the adequacy' of test development and to show which tests 
needed Improvement. 

Distributions of test scores were graphed for students with more 
or less experience. Tests were intended to show successful students 
getting a high percentage of the Items correct. This left most of 
the score range to show deficiencies o^ poorly prepared students. 
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Tn Other Koi^ds, th<^ nta^f attemntpd to develop tests vrith ne{Tatively 
skevmd dIstrlViutions. Any other shaped distributjon shows that test 
r^^vislon was necessary. 

Item analyses were fonductnd to show which items were functioning 
nronerly and to show how others could be improved. These showed 
which items were too easy or too difficult to di^^'erentiate among 
students. They also showed which items differentiate students who 
had learned the basic skills ^rom those who hadn't. For multiple- 
choice tesls, they showed which alternatives aided in the differenti- 
ation and which were either ignored or added confusion, Craig Mueller 
became proficient at using item analysis data to* improve test items. 

tSxaminations for basic skills in three course areas have been 
comnleted and revised. The project has developed tools that are 
useful in identi^^ying students who can benefit from remedial train- 
ing. Tt wou3d be possible* to establish cutoff scores to define 
whi^h students would get special help. Another possibility is that 
pvery student who shows a problem area would receive help even when 
as little as one-half hour is needed to correct the deficiency. 

One objective o^ this project was to select students with de- 
ficiencies for study. They were to be given remediation over an 
extended period and followed up to see its results. However, for 
these students who were already enrolled, remediation was effected 
quickly on specific deficits and the students blended in to larger 
population. The skills tests show great promise for pinpointing 
student de^'icits that need to be treated. 
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'The project sta^f intends to review the test results ^or a 
broader range of students when it can be administered to persons 
be^orp they enroll. They will continue evaluation of the te^Jts 
at that time. However, the present form of the tests will provide 
valuable in^^ormation about student skills without further revision. 

Tn my judgement these tests have shown themselves to be valuable 
in measuring basic skills needed for technical courses. They should 
help to prepare students with skills deficits to succeed in these 
courses, 

Tn addition, the work in this project serves as a model for 
basic skills testing in other courses. I believe it would be valuable 
to compare the findings of this project at RVTI with the situation 
in similar schools. The basic skills required for a given course 
may or may not be similar from one school to another. These tests 
may be useful in other locations in their present or a somewhat modi- 
^•ied form. 

My conclusion is that the objectives of the project are sound 
and that the sta^'' has done a good job in carrying them out. The 
main objective is finding out what a student needs to learn before 
teaching begins. This is so beautifully simple and straightforward 
tnat one wonders why it isn't an integral part of every teaching 
situation. To the credit of the RVTI staff it is becoming a part 
o^ their program. 

The project staff has employed sound procedures in identifying 
and developlnr: measures ''or basic <^kills ^*or a course o'' study. 
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V\(^y are keeping an open mind ia seoi nfj how they can Improve on 
their pre^^ent x)vo(lu<^tr^ amd how they can host be used in the total 
MCat lonal pro^rra^n • 

Respectfully submitted, 



Charles B» Schultz 
Olynpia, Wash. 
(20^) 753-5389 
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SKILLS LIST: 
Identifying Parts of Speech 

1 Nouns 

2 Pronouns 

3 Adjectivej; 

4 Verbs 

5 Adverbs 

6 Prepositions 

7 Conjunctions 

Capitalization 

8 I'irst word of every sentence 

9 Brand names 

10 Building names 

11 Business firms 

12 Days> months 

13 Historical events, periods, 
documents 

14 Institutions 

15 People 

16 School terms 

17 Title of publications 

18 Places 

Identifying Parts of Sentenc e 

19 Subject 

20 Object 

21 Predicate 

Punctuation 

22 Use of comma 

a) Separate words 5 numbers 
in a series 

b) Set off an appositive 

c) After a dependent clause at 
the beginning of a sentence 



ENGLISH 
Punctuation (cont'd 



23 
24 
25 
26 
27 



Before quotations 
With addresses § dates 
Set off such words as 
**of course, indeed, for 
instance, moreover, no 
doubt" 

After introductory words 
which are separated from 
the rest of the sentence 
Before a coordinating con- 
junction in a compound 
sentence 
Quotation marks 
Apostrophe 
Period 

Question mark 
Exclamation point 



d) 
e) 
f) 



g) 



Word Usuage 

28 Correct use of pronouns 

29 Similar sounding words 

30 Correct use of verbs 

Spelling 

31 Common words (20 words) 
Vocabulary: Meaning of Words 

32 Approximate meaning (synonym) 
(25 words) 

Reading 

3.3 ComprcJiension 



Parts of Speech 



SKILLS CHECK LIST 
Punctuation (cont'd) 



1 












I 












3 










4 










5 










6 












7 













Capitalization 









f 








h 








23 












24 














25 












ILL 1 1 














27 













8 




MM 


9 






UL. 
11 






12 






13 
14 






15 








16 








17 








18 









Word Usuage 







1 M 1 1 1 












M M i 



Spelling 

pi M I M iT"i rrr 



Vocabulary 



Parts of a Sentence |32 | | | 








1 1 1 1 1 1 1 1 













19 ■ 


-}- 


20 




2] 


-t- 



Ren ding 

pn-Ti I I 1 1 



,1 . M I i_L.I 



Punctuation 



22a 






b 






c 






dl 
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ANSWER AND SKILL SHEET FOR OFFICE OCCUPATIONS: ENGLISH 



Answer 



Skill/Frequency 



II 



III 



IV. 



VII. 



VIII. 



ERIC 



1) research, findings 

2) or 

3) fine-point, red, yellow 

4) He, we, it 

5) slowly, carefully 

6) believe, arc 

7) In, of to ^ 

8) neither, nor, but 

9) usually, already 

10) electric 

11) students, coursework 

12) During, across 

13) graded, gave 

14) They 



1) 

2) 
3) 
4) 
1) 
4) 



7) 

1) 

2) 

3) 
4) 

5) 
6) 

7) 
fi) 
9) 
10) 
11) 

1) 
2) 
3) 
4) 
5) 
6) 
7) 
8) 
9) 
10) 
•11) 
12) 
13) 



the, £ears, roebuck and co. , 
Braxton builcTing, mid town, 
Wisconsin. 

mr. £ook, kingston £ollege, "£ccu- 
rate t^yping^*, "secretarial careers**, 
during, world war II, stranton 
Technical i^nstTtute ~ 
we, friday, december, s^teve 

Ihe, i Ji JD ^electric, accounting 

Mrs. Tower, 2) duplicator 

3) three instructors 

are required for different corres-' 

pondence. 5) In front of the 

blackboard stood 6) are often 

used by our company. 

English 5 math 8) smile 9) letter 



1/2 . 
7 

3/3 
2/3 
5/2 
4/2 
6/3 
7/3 
5/2 
3 

1/2 
6/2 
4/2 
2 

8, 11, 10, 18/2 



8, 15, 16, 17/2 
8, 13, 14 
8, 12/2, 15 

21/3 



20/3 

25, 24, 24, 22h, 25 
22g, 22a, 25 



Mrs. Miller's, isn* t, working, 
week. 

Yes, businessmen, creditors, gov- 
ernment agencies, managers, well, 
times , pace? 

d idn ' t" address , Mrs. Craftsman, 
school, Ave., Spokane, Washington, 
you'll Mr. Davis' test? 
"You Can Always Take It With You" 
play. 

Unbelievable I exam? 
Mrs. said, "Type... job." 
was, course, leave. 

Wright, teacher, yelled, "Watch out!" 22b, 22d, 23, 27 
semester? 26 



22c, 26 

24, 22c, 25, 22b, 25, 22e, 25 

24, 25, 24, 26 
23, 25 

27, 26 

25, 22d, 23, 25 
22f, 25 



It's 
gone 
we 

eaten, 
than 
which 
me, who 
doesn ' t 
,You ' re 
he,, she 
is 

spoken 
to 



begun 



30 
30 

28 

30/2 

29 

28 

28/2 

30 

28 

28/2 
30 
30 
29 



VT . personnel. Corporation, intention, faculty, review, statement, 
employer , University, recommenda t ion, f 1 exible , forty , twelve , 
Dut ios , s imilar , description , operations , procedures , comparison , 
eligible, truly 33/20 



1-b, 2-d, 3-c, 4-b, 5-a, 6-d, 7-c, 8-a, 9-a, 10-c, 11-d, 12-b, 
13-b, 14-a, 15-a, 16-d, 17-a, 18-c, 19-b, 20-c, 21-b, 22-d, 
23-b, 24-c, 25-c. 



34/25 



Section 1; 1-b, 2-c; 3-c, 4-b, 5-a 
Section II: 1-a, 2-b, 3-d, 4-b, 5-t 
Section III: 1-b, ?-d, 3-c, 4-c, S-il- 
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ANSWKR AND SKILL SHl-ET FOR OFFICE OCCUPATIONS: MATH 



Number Answer Skill/Froquoncy 

1 $70,338.05 39 

2 18S) 34 

3 a) 15t b) 12S% c) 6% 50/3 

4 $949,843.99 37 

5 a) 4.36 b) 2.392 c) .3608 47, 44, 45, 46 
d) .204120 

6 a) .15 b) .061 42/2 

7 a) .25 b) .50 c) .75 d) .80 43/4 

8 a) 25. 5t b) 137t c) 6^ d) 60t 51/4 

9 $198.02 54 

I 0 7 2 wpm 41 

II a) .06 b) .51 c) 3.00 d) .144 52/4 

12 $4729.63 35 

13 a) $1.11 b) 17.109 c) 79,603 39, 34, 38, 40 
d) 188- 

14 718.7316 44 

15 754.00 54 

. 16 a) .098754 b) 1,318 c) 1.170 46, 36, 44, 45 
d) .299 

17 $236 47 

18 74,556 109 38, 40 

19 a) 74.30 b) .7430 48/2 

20 a) 75*. b) 60% c) lOl 53/4 
d) loot 

21 $22.33 54 

22 $162.23 35 

23 a) $16,272.82 b) $158.34 37, 39, 35 
c) $1100.00 

24 38 lbs 40 

25 a) Thirty and one-one hundredth, 42/2 

b) Six hundred twenty-one and 
thirteen thousandths 

26 17.3, 1.73, .173, .0173 .00173 48 

27 $4,717.23 37 

28 $76.44 35 

29 a) .791 b) .347 49/4 

c) .989 d) .138 

30 a) .318 b) 400 c) .333 45, 47, 46, 36 

d) 4205 

31 5,928 36 
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OFFICE OCCUPATIONS SKILLS LIST: MATH 



Basic Calculations 

34 Addition of whole numbers 

35 Addition of money: dollar 
sign and decimal point 

36 Subtraction of whole numbers 

37 Subtraction of money using 
dollar sign and decimal 

38 Multiplication of whole 
numbers 

39 Multiplication of money using 
dollar sign and decimal point 

40 Division of whole numbers 

41 Computing averages 

Decimals 

42 Reading and writing decimals as 
numbers and words 

43 fraction to decimal 

44 Addition of decimals 

45 Subtraction of decimals 

46 Multiplication, of decimals 

47 Division of decimals 

48 Comparing decimals 

49 Rounding Off , 



Percents 

50 Reading and writing percents 

51 Decimal to percent 

52 Percent to decimal 

53 Fraction to percent 

54 Solve for part 

Mechanics and Directions 

55 Mechanics and Directions 



SKILLS PROF I LI- 



Basic Calculations 



Comments : 



.^4 








?5 








36 








57 








3fi 








39 








40 








41 









Decimals 



43 



,44 
!45 

,46 
;47 
.48 
149 



Percents 



SO 




















-S2 










53 










54 











.M echanics a n d Direction s 

iTimiu 
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a> -H ♦J O f-H t-i CO 

V)j=cw60c«6cd4) 
V>3000 "HO 

Cd c B c ojz o *■» 

CJ-iJ-lOO >J=*J'HO 
(d'HUUOCTJ ♦J ^♦J 
dC 0,0) f-H ♦J c 

>sca>x 3f-HCcdT3J-« 
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SKILLS LIST I-OU AUTOMOTIVH MliCHAN 

Uas ic (iompu tat Ions 

1 Addition of whole numbers 

2 Addition of money: dollar 
sign and decimal point 

5 Subtraction of whole numbers 

4 Subtraction of money using 
dollar sign and decimal 

5 Multiplication of whole 
numbers 

6 Multiplication of money 
using dollar sign and 
dec imal point 

7 Division of whole numbers 

8 Division of money using dollar 
sign and decimal point 

9 Computing averages 

Fract ions 

10 Addition of fractions 

11 Subtraction of fractions 

12 Multiplication of fractions 

13 Division of fractions 

14 Reducing fractions 

15 Unlike fractio.ns 



S: MATH, RHADING, VOCABULARY 

16 Fractions to decimals 

17 Fractions to percent 
Decimals 

18 Reading and writing decimals 
as numbers and words 

19 Addition of decimals 

20 Subtraction of decimals 

21 Multiplication of decimals 

22 Division of decimals 

23 Comparing decimals 

24 Rounding off 

25 Decimal to percent 
Percentage 

26 I'ercent to decimaT 

27 Percent of a total 
Reading 

28 Comprehension 
Vocabulary 

29 Approximate meaning (synonym) 
Mechanics and Directions 
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SKILLS PROFILE 



Basic Computations 



1 








2 








3 








4 








5 








6 








7 








8 








9 









Per centage 









1 1 


V 









Readin 
!Z8 



fjjjj J-lTTTITTn-l 



Voca bulary 



Ul 1 








1 


111 


















13! 
















15; 








16' 








17' 









P9 




















1 1 1 1 1 1 

























Mechanics and Directions 

[ 3oi 1 1 1 1 i xm 



Deci mals 



111 










19 
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20 










71 










22 










23 










124 








25 







131 



ANSWER AND SKILL SHEET FOR AUTOMOTIVE MECHANICS: MATH, 



READING, VOCABULARY 



Number Answer 



Skills/Frequency 



1 
2 
3 
4 
5 
6 
7 

8 

9 
10 
11 
12 
13 
14 

15 
16 

17 



18 
19 
20 
21 

22 

23 
24 
25 
26 
27 

28 
29 
30 
31 
32 
33 
34 
35 

36 



c) 13/16 d) 5/27 



d) 4/5 



or 



b) 4706 c) 1,406 d) 93 



$59.45 

a) 7/8 b) 3/10 
225.45 miles 
7704 8560.8560 
a) 2/3 b) 5/9 c) 13/16 
85 

a) 3/14 b) 3/16 c) 44/80 
11/20 d) 5/8 

a) 2S\ b) 3961 c) 31 d) 401 
.014 
2495 hp 
a) 67095 
$25.40 
5524 

a) .096 b) 1.748 c) .382620 
d) .96 
$ 4412.20 
a) 17/20 b) 2/15 
5/14 

six hundred thirteen thousandths 18/3 
fifteen and ninety- three hundredths 
one thousand four hundred seventy-six 
ten thousandths 

88 

$218.10 

39.6, 3.96, .396, .0396, .00396 
a) .39326 b) .562 c) .9993 
.032 

21/80 b) 1/2 c) 105/384 
5/6 

c) 60% d) 1001 



2 

10/2. 11/2 
21 

23>2 20, 19 
14/4 
3 

12/2, 13/2 

25/4 
20 
1 

5.1.3,7 

27 
7 

20, 19, 21, 22 



d) 
a) 
b) 
c) 



c) 8/8< or 1 



601, b) 504 
4 p. s. i. 



30 
b) 



d) 
a) 
d) 
a) 
22 
55 

a) .75 b) . 
a) $1002.00 
d) $1,093.11 
$297.80 
12 

13/16 
45.05 
23.5 
$81.23 

a) 1.50 b) .33 
a) 5/8 b) 21/24 
d) 34/64 or 17/32 
a) .82 b) .57 c) 



6 

15/4 



27 
6 

23 

21, 20, 19, 22 



12/2, 13/2 



c) .50 d) 1.00 
$9.67 c) $632.44 



c) .07 
c) 2/7 



d) .145 



.43 d) .79 



17/4 
19 
9 

16/4 

2. 8, 6, 4 
4 

22 
10 
21 
22 
8 

26/4 

10/2, 11/2 
24/4 



READING; 

Section I: 
Section II: 
Section III: 

VOCABULARY: 



1-c, 2-c, 3-d, 4-d, 5-c 
1-c, 2-c, 3-d, 4-b, 5-b 
1-b, 2-d, 3-c, 4-b, 5-a 



1-b, 2-c, 3-b, 4-d, 5-a, 6-a, 7-a, 8-c, 9-a, 10-a 
11-c, 12-d, 13-c, 14-a, 15-c, 16-a, 17-b, 18-d, 19-b, 
20-c, 21-d, 22-c, 2*^-d, 24-b, 25-c 
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AtiSWER AND SKILL SHERT FOR MACHINE SHOP: MATH, READING 

VOCABULARY ^ ^^^^^ ' 



Number Answer Skills/Frequ ency 



1 


a) 537 b) 1343 c) 991 
d) 4876 


4, 1, 2, 3 


2 


a) 7 b) 1.082 c) 9.819 


13, 10, 11, 12 




d) .272824 

X » 1 13/16»» Y « 1 3/8" 




3 


6, 5 


4 


1.026" 


12 


5 


353 


4 


6 


.108 


10 ' 


7 


3898 


2 


8 


Largest .100, Smallest .003 


15 


9 


a) 5/6 b) 5/8 c) 3/16 d) 3/5 


5, 6, 7, 8 


10 


1252 


1 


11 


a) .004 b) .590 b) .364 
d) .893 


16/4 


12 


1.249 


13 


13 


X = 2 2/8", Y - 27/16" or 1 11/16" 
a) 3.635 b) .00512 c).2095 d)1.5558 


6, 5 


14 


13, 12,11, 10 


15 


11,256 


3 


16 


.208 


11 


17 


96 


4 


18 


a) 5 5/12 b) 2/3 


7, 8 


19 


57* 


1 


20 


.00144, .0144, .144, 1.44, 14.4 


15 


21 


10 2/7 times 


8 


22 


a) 470 b) 49,900 c) 192 


4, 1, 2, 3 




d) 154,224 


9/4 


23 


a) .8 b) .7 c) .75 d) .25 


24 


63" 


7 


25 


342.88+. 25 - 343.13 


12 


26 


a) 1 6/8 or 1 3/4 b) 3/4 c) 13/16 
d) 16/35 


8, 5, 6, 7 


27 


16 


13 


28 


1332 


3 


29 


.310 


10 


30 


2,091 


2 


31 


8180 


1 


32 


B.508 


15 


33 


. 1780 


11 


34 


a) 3/4 b) 3/8 c) 7/8 d) 3/10 


14/4 



Reading 

Section I: 1-d, 2-c, 3-c, 4-a, 5-b 

Section II: 1-a, 2-c, 3-d, 4-c, 5-b 

Section III: 1-c, 2-b, 3-b, 4-c, 5-d 

Vocabulary: 

I- d, 2-a, 3-b, 4-c, 5-d, 6-a, 7-d, 8-d, 9-c, 10-c, 

II- b, 12-a, 13-b, 14-a, 15-d, 16-c, 17-a, 18-a, 
19-c, 20-c, 21-d, 22-d, 23-b, 24-a, 25-d 



SKILLS LIST FOR MACHINE SHOP: MATH, READING, VOCABULARY 



Basic Calculations 

1 Addition of whole numbers 

2 Subtraction of whole numbers 

3 Multiplication of whole numbers 

4 Division of whole numbers 

Fractions 

5 Addition of fractions 

6 Subtraction of fractions 

7 Multiplication of fractions 

8 Division of fractions 

9 Fractions to decimals 

Decimals 

10 Addition of decimals 

11 Subtraction of decimals 

12 Multiplication of decimals 

13 Division of decimals 



Decimals (contd) 

14 Decimals to fractions 

15 Comparing decimals 

16 Rounding off 

Reading 

17 Comprehension 
Vocabulary 

18 Approximate meaning (synonym) 
Mechanics and Directions 

19 Mechanics and directions 



SKILLS PROFILE 



Basic Calculations 



J. 
4 



Fractions 



5 










6 










7 










n 










9 











Dec imals 



10 




















12 










13 










14 










J5 










16 











Reading 

bii n 1 1 I 1 1 TT 

Vocabulary 

bl I I I I I I I I 11 

Mechanics G Directions 

hoi i i i i I I I I 



Comments: 
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